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Middle College High School 



Program description Middle College High Schools are alternative high schools 

located on college campuses that aim to help at-risk students 
complete high school and encourage them to attend col- 
lege. The schools offer a project-centered, interdisciplinary 



curriculum, with an emphasis on team teaching, individualized 
attention, and development of critical thinking skills. Students 
are also offered support services, including specialized counsel- 
ing, peer support, and career experience opportunities. 



Research One study of Middle College High School met the What Works 
Clearinghouse (WWC) evidence standards. This randomized 
controlled trial included 394 students in the Seattle Public 
Schools who were assigned to an intervention group that was 
offered admission to the alternative high school or a control 



group that was not. Control group students were free to par- 
ticipate in other regular and alternative high schools operated 
by the school district and in General Educational Development 
(GED) programs. Most control group students participated in 
one of these other education options. 1 



Effectiveness Middle College High School was found to have no discernible effects on staying in school or completing school. 



Staying in school Progressing in school Completing school 



Rating of effectiveness No discernible effects na 

Improvement index 2 Average: -3 percentile points na 



No discernible effects 
Average: +2 percentile points 



na = not applicable 



1. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average improvement index for all findings across the study. 
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Research 



WWC Intervention Report 



The Middle College High School study summarized in this 
intervention report was prepared by staff of Mathematica Policy 
Research, Inc. (MPR). Because the principal investigator for the 
WWC dropout prevention review is also an MPR staff member 
and the lead study author, the study was rated by staff members 



from Caliber, an ICF International Company, who also prepared 
the intervention report. The report was then reviewed by MPR 
staff members and by members of the WWC Technical Review 
Team and external peer reviewers. 



Developer and contact 

Information on the history of the Middle College High School 
model and current resources for program implementation are 
available from the Middle College National Consortium (MCNC). 
Web: http://www.mcnc.us. Telephone: (718) 361-1981. 

Scope of use 

The MCNC reports that, as of December 2006, the Middle Col- 
lege High School program was operating in 31 school districts in 
12 states. 

Description of intervention 

Middle College High Schools are alternative high schools that 
operate as formal collaborations between local school districts 
and colleges. The schools, which offer regular high school diplo- 
mas, are small— with fewer than 100 students per grade— and 
are located on college campuses. Faculty and students have 
access to the college’s educational resources and facilities, and 
students can take college-level courses. The curriculum empha- 
sizes development of critical thinking skills and connecting what 
is learned to real-world experiences. These schools typically 



offer career-oriented courses and internships. In addition, 
students often must complete a community service requirement 
to graduate. Classes are taught by high school teachers from the 
local school district. Faculty teach collaboratively and integrate 
material across disciplines. Within team-taught classes, students 
often participate in collaborative learning groups. Student-to- 
staff ratios are substantially lower than in traditional high school 
programs, allowing more individual attention. Middle College 
High Schools often use alternative assessment strategies, such 
as portfolios and oral presentations. They emphasize democratic 
school governance and use school committees— including 
administrators, faculty, parents, students, and college and 
community representatives— to provide input and guidance on 
school operations. 

Cost 

Researchers estimated the cost of Middle College High 
School in Seattle to be $965 a student per month of program 
participation— about 50% higher than the cost of educating a 
student in a regular school within the district (estimated to be 
$649 a month). 3 



The WWC reviewed six studies of the effectiveness of Middle 
College High School. One study (Dynarski, Gleason, Rangarajan, 
& Wood, 1998) was a randomized controlled trial that met WWC 
evidence standards. The other five studies did not meet WWC 
evidence screens. 



The Dynarski et al. (1998) study of Middle College High 
School was part of a larger evaluation examining the effective- 
ness of 16 middle school and high school dropout prevention 
programs. The Middle College High School study used a ran- 
dom assignment design and included 395 students who applied 



3. See Rosenberg, L., & Hershey, A. (1995). The cost of dropout prevention programs. Princeton, NJ: Mathematica Policy Research, Inc. Costs have been 
converted to 2006 dollars using the Consumer Price Index. 
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Research (continued) 



Effectiveness 



The WWC found Middle 
College High School to 
have no discernible effects 
on staying in school or 
completing school 
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to attend the alternative high school, which was operated by 
Seattle Public Schools in cooperation with Seattle Central 
Community College. Study participants were generally older 
students— their average age was just under 18— who were over- 
age for grade or had dropped out of school. Students assigned 
to the control group did not receive Middle College High School 



services, but they were free to participate in other regular and 
alternative education programs in the community. Most control 
group students participated in one of these other education 
options. Findings presented in this report were drawn from a 
follow-up survey administered about two years after random 
assignment. 4 



Findings 

The WWC review of interventions for dropout prevention 
addresses student outcomes in three domains: staying in 
school, progressing in school, and completing school. The 
Dynarski, Gleason, Rangarajan, and Wood (1998) study exam- 
ined outcomes in two of these domains. 

Staying in school. Dynarski et al. (1998) reported that by 
the end of the second year after random assignment, 36% of 
students in the Middle College High School group had dropped 
out of school, compared with 33% of control group students— a 
difference that was not statistically significant. In addition, this 
difference was not large enough to be considered substantively 
important based on WWC standards. 

Completing school. Dynarski et al. (1998) found that 40% of 
students in the Middle College High School group had earned a 



high school diploma or GED certificate two years after random 
assignment, compared with 38% of control group students— a 
difference that was not statistically significant or substantively 
important. 5 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given 
outcome domain as positive, potentially positive, mixed, no 
discernible effects, potentially negative, or negative. The rating 
of effectiveness takes into account four factors: the quality 
of the research design, the statistical significance of the find- 
ings, 6 the size of the difference between participants in the 
intervention and comparison conditions, and the consistency 
in findings across studies (see the WWC Intervention Rating 
Scheme) . 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technical 
Details of WWC-Conducted Computations) . The improvement 



index represents the difference between the percentile rank 
of the average student in the intervention condition versus 
the percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 
index is entirely based on the size of the effect, regardless of 
the statistical significance of the effect, the study design, or the 



4. An additional follow-up survey was conducted at the end of year three with an early cohort of study participants. Because of relatively low response rates 
to this survey, as well as evidence of substantial intervention-control differences in baseline characteristics among respondents, these third-year results 
were not used in the WWC rating of the effectiveness of Middle College High School. These results are summarized in Appendices A4.2 and A4.3. 

5. In addition, analysis of third-year survey data, available for an early cohort, indicates no statistically significant effect of the intervention on completing 
school after three years. However, these longer-term results suggest that Middle College High School may have shifted these completions toward receipt 
of regular high school diplomas and away from receipt of GED certificates. Appendix A4.3 presents these longer-term results. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted 
Computations for the formulas the WWC used to calculate the statistical significance. In the case of Middle College High School, no corrections for 
clustering or multiple comparisons were needed. 
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The WWC found Middle 
College High School to 
have no discernible effects 
on staying in school or 
completing school (continued) 



analyses. The improvement index can take on values between 
-50 and +50, with positive numbers denoting results favorable to 
the intervention group. 

The average improvement index was -3 percentile points for 
staying in school and +2 percentile points for completing school 
in the one study that passed WWC evidence screens. 



Summary 

The WWC reviewed six studies on Middle College High School. 
One study met WWC standards; the others did not meet WWC 
evidence screens. Based on this one study, the WWC found the 
intervention to have no discernible effects on staying in school 
or completing school. The evidence presented in this report is 
limited and may change as new research emerges. 
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For more information about specific studies and WWC calculations, please see the WWC Middle Collette High 
School Technical Appendices . 



7. The study did not use a comparison group. 

8. Lack of evidence of baseline equivalence: the study, which used a quasi-experimental design, did not establish that the comparison and intervention 
groups were equivalent at baseline. 

9. The outcome measures examined in this study are not relevant to this review. 
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Appendix A1 


Study characteristics: Dynarski, Gleason, Rangarajan, & Wood, 1998 (randomized controlled trial) 


Characteristic 


Description 


Study citation 


Dynarski, M., Gleason, P., Rangarajan, A., & Wood, R. (1998). Impacts of dropout prevention programs: Final report. A research report from the School Dropout Demonstration 
Assistance Program evaluation. Princeton, NJ: Mathematica Policy Research, Inc. 


Participants 


The Middle College High School study used a randomized controlled trial research design. The original study sample of 516 students was comprised of two cohorts. Cohort 
1 included 199 students in the intervention group and 123 students in the control group who applied to attend the alternative high school at the beginning of the 1992-93 
school year. Cohort 2 included 123 students in the intervention group and 71 students in the control group who applied to attend the alternative high school at the beginning 
of the 1993-94 school year. 

Participants were, on average, 18 years old at the time they applied to Middle College High School. About half of the sample was African-American; about one in five was 
white; about one in 12 was Hispanic; and slightly fewer than one in four was Asian or other ethnicities. Participants were evenly split between males and females. Most were 
behind grade level at baseline; two-thirds had had discipline problems at school in the past year; and just over half had dropped out of school in the past. 

Results summarized here are drawn from a follow-up survey administered two years after random assignment; 244 intervention group students and 150 control group 
students responded — for response rates of 76% and 77%, respectively. 1 Researchers compared the baseline characteristics of the two research groups on 13 demographic, 
socioeconomic, and school performance measures. A statistical test of the overall difference between the follow-up survey respondents in the two research groups on the full 
set of 13 baseline characteristics found that the groups were not significantly different. 

An additional follow-up survey was conducted three years after random assignment with cohort 1 only. Because of relatively low response rates and evidence of substantial 
intervention-control baseline differences among respondents, these third-year results were not used for the WWC effectiveness rating of Middle College High School. These 
longer-term results are summarized in Appendices A4.2 and A4.3. 


Setting 


The Middle College High School study was conducted at an alternative high school on the campus of the Seattle Central Community College in Seattle, Washington. 


Intervention 


Seattle's Middle College High School, which opened in 1990 and continues to operate, targets students who are close to dropping out or who have dropped out in the past. 

It is run as a collaboration between the Seattle Public Schools and the Seattle Central Community College. It offers students the opportunity to earn a high school diploma in 
a small alternative school located on the community college campus. The school emphasizes the development of critical thinking skills and focuses on experiential learning, 
internships, and support services. Services for students include individual counseling, peer support groups, attendance monitoring, and career awareness. In addition, 
students can take community college courses and use the college’s academic and sports facilities. Students are active in school governance and operations. They participate 
in screening new applicants, running assembly programs, and formulating school policies on attendance, discipline, and other issues. 

At the time it was evaluated, the school enrolled about 300 students and its core academic curriculum focused on two modules — math/science and integrated humanities. 
Each module was taught by a team of two teachers, supported by two or three paid, in-class tutors. Within these team-taught classes, students frequently participated 
in collaborative learning groups and worked on projects with other students. Each quarter, teachers developed a curriculum around a unifying theme, such as “rights and 
responsibilities” (Hershey, Adelman, & Murray, 1995). 




(continued) 
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Study characteristics: Dynarski, Gleason, Rangarajan, & Wood, 1998 (randomized controlled trial) (continued) 


Characteristic 


Description 


Comparison 


The control group did not receive Middle College High School services, but they were free to participate in other regular and alternative education programs in the community. 
Most control group students participated in one of these other education options. During the first year after random assignment control group members reported spending 
63% of their time enrolled in a school or GED program, on average, compared with 60% for intervention students. According to student self-reports, more than a third of the 
time control group members spent attending a school or GED program was spent attending one of Seattle's other alternative high schools. 


Primary outcomes 
and measurement 


Two relevant outcomes from the Middle College High School study are included in this review: whether participants dropped out of school and whether they earned a high 
school diploma or GED certificate. (See Appendices A2.1 and A2.2 for a more detailed description of these outcome measures.) 


Teacher training 


Middle College High School teachers were regular high school teachers employed by the Seattle Public Schools. No additional information about their specific training was 
available. 



1. Although the overall second-year response rates for the two cohorts combined were similar for intervention and control group students, there were substantial intervention-control differences in 
response rates within each cohort— particularly cohort 2. For cohort 1, the response rate for intervention group students was 82% and for control group students 75%. For cohort 2, the response 
rate for intervention group students was 65% and for control group students 82%. 
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Appendix A2.1 Outcome measures for the staying in school domain 



Outcome measure 


Description 


Dropped out 


Percentage of students who dropped out of school by the end of the second follow-up year. These self-reported data were collected from follow-up surveys. 



Appendix A2.2 Outcome measures for the completing school domain 



Outcome measure 


Description 


Earned a high school 
diploma or GED 


Percentage of students who received a high school diploma or GED certificate by the end of the second follow-up year. These self-reported data were collected from follow-up 
surveys. 
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Appendix A3.1 



Summary of study findings included in the rating for the staying in school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dropped out (%) Cohorts 1 and 2 


394 


Dynarski et al., 1998 (randomized controlled trial) 6 

36 33 -3 


-0.08 


ns 


-3 


Domain average 7 for staying in school 






-0.08 


ns 


-3 



ns = not statistically significant 

1 . This appendix reports second-year follow-up findings considered for the effectiveness rating and the average improvement index. Third-year follow-up findings — available for cohort 1 only — are not included in these ratings but are 
reported in Appendix A4.2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the mean difference, effect size, and improve- 
ment index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculations, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Dynarski et al. (1998), no corrections for 
clustering or multiple comparisons were needed. 

7. This row provides the study average, which in this case is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 







Dynarski et al., 1998 (randomized controlled trial) 6 








Earned a high school Cohorts 1 and 2 


394 


40 38 2 


0.05 


ns 


+2 


diploma or GED (%) 












Domain average 7 for completing school 






0.05 


ns 


+2 



ns = not statistically significant 

1 . This appendix reports second-year follow-up findings considered for the effectiveness rating and the average improvement index. Third-year follow-up findings — available for cohort 1 only — are not included in these ratings but are 
reported in Appendix A4.3. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculations, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Dynarski et al. (1998), no corrections for 
clustering or multiple comparisons were needed. 

7. This row provides the study average, which in this case is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A4.1 



Summary of additional findings for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dynarski et al., 1998 (randomized controlled trial) 6 



Earned a high school 
diploma (%) 


Cohorts 1 and 2 


394 


21 


18 


3 


0.12 


ns 


+5 


Earned a GED certificate (%) 


Cohorts 1 and 2 


394 


18 


20 


-2 


-0.08 


ns 


-3 



ns = not statistically significant 

1 . This appendix presents findings for the intervention's separate effects on high school diploma and GED certificate receipt. The intervention’s effect on the combined measure of high school diploma or GED receipt was used for determin- 
ing the effectiveness rating and is presented in Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 



WWC Intervention Report Middle College High School 



March 12, 2007 



10 





Appendix A4.2 Summary of longer-term subgroup findings for the staying in school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dropped out at end 


Cohort 1 only 


217 


Dynarski et al., 1998 (randomized controlled trial) 6 

31 38 7 


0.19 


ns 


+7 


of third year (%) 















ns = not statistically significant 

1 . This appendix presents the third-year follow-up findings for measures in the staying in school domain. These findings were not used for intervention rating purposes because the third-year survey was administered to cohort 1 only and 
because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and control group members who responded to the third-year survey. The intervention’s effect on 
staying in school was rated based on the second-year follow-up findings presented in Appendix A3.1 . 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the mean difference, effect size, and improve- 
ment index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 
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Appendix A4.3 Summary of longer-term subgroup findings for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dynarski et al., 1998 (randomized controlled trial) 6 



Earned a high school diploma 
by end of third year (%) 


Cohort 1 only 


217 


31 


23 


8 


0.25 


ns 


+10 


Earned a GED certificate 
by end of third year (%) 


Cohort 1 only 


217 


24 


37 


-13 


-0.38 


Statistically 

significant 


-15 


Earned a high school diploma 
or GED by end of third year (%) 


Cohort 1 only 


217 


55 


61 


-6 


-0.15 


ns 


-6 



ns = not statistically significant 

1 . This appendix presents the third-year follow-up findings for measures in the completing school domain. These findings were not used for intervention rating purposes because the third-year survey was administered to cohort 1 only and 
because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and control group members who responded to the third-year survey. The intervention's effect on 
completing school was rated based on the second-year follow-up findings presented in Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 
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Appendix A5.1 Middle College High School rating for the staying in school domain 



The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of staying in school, the WWC rated Middle College High School as having no discernible effects. It did not meet the criteria for other 
ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the one study that met WWC evidence 
standards did not show statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. The only study of Middle College High School that met WWC evidence standards found no statistically significant or substantively important 
effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Middle College High School had only one study meeting WWC evidence standards. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important effects, either positive or negative, in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies found statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence 

(continued) 
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Appendix A5.1 Middle College High School rating for the staying in school domain (continued) 



• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies found statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 



WWC Intervention Report Middle College High School 



March 12, 2007 



14 








Appendix A5.2 Middle College High School rating for the completing school domain 



The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of completing school, the WWC rated Middle College High School as having no discernible effects. It did not meet the criteria for other 
ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the one study that met WWC evidence 
standards did not show statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. No studies found statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Middle College High School had only one study meeting WWC evidence standards. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important effects, either positive or negative, in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies found statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence 

(continued) 
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Appendix A5.2 Middle College High School rating for the completing school domain (continued) 



• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies found statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix 



Appendix A1 


Study characteristics: Dynarski, Gleason, Rangarajan, & Wood, 1998 (randomized controlled trial) 


Characteristic 


Description 


Study citation 


Dynarski, M., Gleason, P., Rangarajan, A., & Wood, R. (1998). Impacts of dropout prevention programs: Final report. A research report from the School Dropout Demonstration 
Assistance Program evaluation. Princeton, NJ: Mathematica Policy Research, Inc. 


Participants 


The Middle College High School study used a randomized controlled trial research design. The original study sample of 516 students was comprised of two cohorts. Cohort 
1 included 199 students in the intervention group and 123 students in the control group who applied to attend the alternative high school at the beginning of the 1992-93 
school year. Cohort 2 included 123 students in the intervention group and 71 students in the control group who applied to attend the alternative high school at the beginning 
of the 1993-94 school year. 

Participants were, on average, 18 years old at the time they applied to Middle College High School. About half of the sample was African-American; about one in five was 
white; about one in 12 was Hispanic; and slightly fewer than one in four was Asian or other ethnicities. Participants were evenly split between males and females. Most were 
behind grade level at baseline; two-thirds had had discipline problems at school in the past year; and just over half had dropped out of school in the past. 

Results summarized here are drawn from a follow-up survey administered two years after random assignment; 244 intervention group students and 150 control group 
students responded — for response rates of 76% and 77%, respectively. 1 Researchers compared the baseline characteristics of the two research groups on 13 demographic, 
socioeconomic, and school performance measures. A statistical test of the overall difference between the follow-up survey respondents in the two research groups on the full 
set of 13 baseline characteristics found that the groups were not significantly different. 

An additional follow-up survey was conducted three years after random assignment with cohort 1 only. Because of relatively low response rates and evidence of substantial 
intervention-control baseline differences among respondents, these third-year results were not used for the WWC effectiveness rating of Middle College High School. These 
longer-term results are summarized in Appendices A4.2 and A4.3. 


Setting 


The Middle College High School study was conducted at an alternative high school on the campus of the Seattle Central Community College in Seattle, Washington. 


Intervention 


Seattle's Middle College High School, which opened in 1990 and continues to operate, targets students who are close to dropping out or who have dropped out in the past. 

It is run as a collaboration between the Seattle Public Schools and the Seattle Central Community College. It offers students the opportunity to earn a high school diploma in 
a small alternative school located on the community college campus. The school emphasizes the development of critical thinking skills and focuses on experiential learning, 
internships, and support services. Services for students include individual counseling, peer support groups, attendance monitoring, and career awareness. In addition, 
students can take community college courses and use the college’s academic and sports facilities. Students are active in school governance and operations. They participate 
in screening new applicants, running assembly programs, and formulating school policies on attendance, discipline, and other issues. 

At the time it was evaluated, the school enrolled about 300 students and its core academic curriculum focused on two modules — math/science and integrated humanities. 
Each module was taught by a team of two teachers, supported by two or three paid, in-class tutors. Within these team-taught classes, students frequently participated 
in collaborative learning groups and worked on projects with other students. Each quarter, teachers developed a curriculum around a unifying theme, such as “rights and 
responsibilities” (Hershey, Adelman, & Murray, 1995). 




(continued) 
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Appendix A1 


Study characteristics: Dynarski, Gleason, Rangarajan, & Wood, 1998 (randomized controlled trial) (continued) 


Characteristic 


Description 


Comparison 


The control group did not receive Middle College High School services, but they were free to participate in other regular and alternative education programs in the community. 
Most control group students participated in one of these other education options. During the first year after random assignment control group members reported spending 
63% of their time enrolled in a school or GED program, on average, compared with 60% for intervention students. According to student self-reports, more than a third of the 
time control group members spent attending a school or GED program was spent attending one of Seattle's other alternative high schools. 


Primary outcomes 
and measurement 


Two relevant outcomes from the Middle College High School study are included in this review: whether participants dropped out of school and whether they earned a high 
school diploma or GED certificate. (See Appendices A2.1 and A2.2 for a more detailed description of these outcome measures.) 


Teacher training 


Middle College High School teachers were regular high school teachers employed by the Seattle Public Schools. No additional information about their specific training was 
available. 



1. Although the overall second-year response rates for the two cohorts combined were similar for intervention and control group students, there were substantial intervention-control differences in 
response rates within each cohort— particularly cohort 2. For cohort 1, the response rate for intervention group students was 82% and for control group students 75%. For cohort 2, the response 
rate for intervention group students was 65% and for control group students 82%. 
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Appendix A2.1 Outcome measures for the staying in school domain 



Outcome measure 


Description 


Dropped out 


Percentage of students who dropped out of school by the end of the second follow-up year. These self-reported data were collected from follow-up surveys. 



Appendix A2.2 Outcome measures for the completing school domain 



Outcome measure 


Description 


Earned a high school 
diploma or GED 


Percentage of students who received a high school diploma or GED certificate by the end of the second follow-up year. These self-reported data were collected from follow-up 
surveys. 
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Appendix A3.1 



Summary of study findings included in the rating for the staying in school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dropped out (%) Cohorts 1 and 2 


394 


Dynarski et al., 1998 (randomized controlled trial) 6 

36 33 -3 


-0.08 


ns 


-3 


Domain average 7 for staying in school 






-0.08 


ns 


-3 



ns = not statistically significant 

1 . This appendix reports second-year follow-up findings considered for the effectiveness rating and the average improvement index. Third-year follow-up findings — available for cohort 1 only — are not included in these ratings but are 
reported in Appendix A4.2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the mean difference, effect size, and improve- 
ment index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculations, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Dynarski et al. (1998), no corrections for 
clustering or multiple comparisons were needed. 

7. This row provides the study average, which in this case is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 







Dynarski et al., 1998 (randomized controlled trial) 6 








Earned a high school Cohorts 1 and 2 


394 


40 38 2 


0.05 


ns 


+2 


diploma or GED (%) 












Domain average 7 for completing school 






0.05 


ns 


+2 



ns = not statistically significant 

1 . This appendix reports second-year follow-up findings considered for the effectiveness rating and the average improvement index. Third-year follow-up findings — available for cohort 1 only — are not included in these ratings but are 
reported in Appendix A4.3. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculations, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Dynarski et al. (1998), no corrections for 
clustering or multiple comparisons were needed. 

7. This row provides the study average, which in this case is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A4.1 



Summary of additional findings for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dynarski et al., 1998 (randomized controlled trial) 6 



Earned a high school 
diploma (%) 


Cohorts 1 and 2 


394 


21 


18 


3 


0.12 


ns 


+5 


Earned a GED certificate (%) 


Cohorts 1 and 2 


394 


18 


20 


-2 


-0.08 


ns 


-3 



ns = not statistically significant 

1 . This appendix presents findings for the intervention's separate effects on high school diploma and GED certificate receipt. The intervention’s effect on the combined measure of high school diploma or GED receipt was used for determin- 
ing the effectiveness rating and is presented in Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 
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Appendix A4.2 Summary of longer-term subgroup findings for the staying in school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dropped out at end 


Cohort 1 only 


217 


Dynarski et al., 1998 (randomized controlled trial) 6 

31 38 7 


0.19 


ns 


+7 


of third year (%) 















ns = not statistically significant 

1 . This appendix presents the third-year follow-up findings for measures in the staying in school domain. These findings were not used for intervention rating purposes because the third-year survey was administered to cohort 1 only and 
because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and control group members who responded to the third-year survey. The intervention’s effect on 
staying in school was rated based on the second-year follow-up findings presented in Appendix A3.1 . 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the mean difference, effect size, and improve- 
ment index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 
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Appendix A4.3 Summary of longer-term subgroup findings for the completing school domain 1 









Author’s findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean difference 2 
(Middle College 
High School - 
comparison) 


Statistical 
significance 4 
Effect size 3 (at a = 0.05) 


Improvement 

index 5 



Dynarski et al., 1998 (randomized controlled trial) 6 



Earned a high school diploma 
by end of third year (%) 


Cohort 1 only 


217 


31 


23 


8 


0.25 


ns 


+10 


Earned a GED certificate 
by end of third year (%) 


Cohort 1 only 


217 


24 


37 


-13 


-0.38 


Statistically 

significant 


-15 


Earned a high school diploma 
or GED by end of third year (%) 


Cohort 1 only 


217 


55 


61 


-6 


-0.15 


ns 


-6 



ns = not statistically significant 

1 . This appendix presents the third-year follow-up findings for measures in the completing school domain. These findings were not used for intervention rating purposes because the third-year survey was administered to cohort 1 only and 
because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and control group members who responded to the third-year survey. The intervention's effect on 
completing school was rated based on the second-year follow-up findings presented in Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Dynarski et al. (1998), no correction for clustering was needed. 
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Appendix A5.1 Middle College High School rating for the staying in school domain 



The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of staying in school, the WWC rated Middle College High School as having no discernible effects. It did not meet the criteria for other 
ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the one study that met WWC evidence 
standards did not show statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. The only study of Middle College High School that met WWC evidence standards found no statistically significant or substantively important 
effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Middle College High School had only one study meeting WWC evidence standards. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important effects, either positive or negative, in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies found statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence 

(continued) 
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Appendix A5.1 Middle College High School rating for the staying in school domain (continued) 



• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies found statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 Middle College High School rating for the completing school domain 



The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of completing school, the WWC rated Middle College High School as having no discernible effects. It did not meet the criteria for other 
ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the one study that met WWC evidence 
standards did not show statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. No studies found statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Middle College High School had only one study meeting WWC evidence standards. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important negative effects in this domain. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies found statistically significant or substantively important effects, either positive or negative, in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies found statistically significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence 

(continued) 
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Appendix A5.2 Middle College High School rating for the completing school domain (continued) 



• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies found statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies found statistically significant or substantively important positive effects in this domain. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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